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DETAILED ACTION 

Drawing Objections 

The drawings are objected to under 37 CFR 1.83(a). The drawings must show 
every feature of the invention specified in the claims. Therefore, the complete features 
of claims 7 (converting speech to text and prosody), 9 (processor), and 13-14 must be 
shown or the feature(s) canceled from the claim(s). No new matter should be entered. 

Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in 
reply to the Office action to avoid abandonment of the application. Any amended 
replacement drawing sheet should include all of the figures appearing on the immediate 
prior version of the sheet, even if only one figure is being amended. The figure or figure 
number of an amended drawing should not be labeled as "amended." If a drawing figure 
is to be canceled, the appropriate figure must be removed from the replacement sheet, 
and where necessary, the remaining figures must be renumbered and appropriate 
changes made to the brief description of the several views of the drawings for 
consistency. Additional replacement sheets may be necessary to show the renumbering 
of the remaining figures. Each drawing sheet submitted after the filing date of an 
application must be labeled in the top margin as either "Replacement Sheet" or "New 
Sheet" pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, 
the applicant will be notified and informed of any required corrective action in the next 
Office action. The objection to the drawings will not be held in abeyance. 
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Claim Rejections - 35 USC § 101 

1. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

2. Claims 15-18 are rejected under 35 U.S.C. 101 because the claimed invention is 
directed to non-statutory subject matter. 

With respect to claims 15-18, the "computer readable medium," in accordance 
with paragraph s 0049 and 0056 of applicant's specification, may be an "optical media" 
(i.e. a electromagnetic signal). This subject matter is not limited to that which falls within 
a statutory category of invention because it is not limited to a process, machine, 
manufacture, or a composition of matter. Instead, it includes a form of energy. Energy 
does not fall within a statutory category since it is clearly not a series of steps or acts to 
constitute a process, not a mechanical device or combination of mechanical devices to 
constitute a machine, not a tangible physical article or object which is some form of 
matter to be a product and constitute a manufacture, and not a composition of two or 
more substances to constitute a composition of matter. See Interim Guidelines on 35 
USC 101. Annex IV: Computer-Related Nonstatutory Subject Matter. 

Claim Rejections - 35 USC § 102 

3. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
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granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21 (2) 
of such treaty in the English language. 

4. Claims 1-6, 8-12, and 14-18 are rejected under 35 U.S. C. 102(e) as being 

anticipated by Bhaskaran et al., US 2004/0141630. 

Re claim 1, Bhaskaran et al. discloses a picture, comprising: a hard-copy 
medium (see paragraphs 0003 & 0033); and pigment (i.e. audio augmented image data, 
see element 122 in figure 3), imprinted on the hard-copy medium so as to define an 
image incorporating markings that are substantially imperceptible to an unaided eye of a 
human viewer and that encode audio data associated with the image (see figure 3, and 
paragraphs 0003, 0033, 0036-0037). 

Re claim 2, Bhaskaran further discloses the pigment is imprinted on the hard- 
copy medium so as to define dots (i.e. halftone dots or pixels) of varying sizes (i.e. 
modifying) within respective cells (i.e. block or patch) (see paragraphs 0034-0035), and 
wherein the audio data (i.e. embedding) are encoded in the picture by varying 
respective positions of the dots within the respective cells (see figure 9, and paragraphs 
0048-0052). 

Re claim 3, Bhaskaran further discloses a method for encoding information (see 
paragraph 0032), comprising: capturing an image of a subject so as to generate image 
data (see element 108, figure 3; paragraph 0036); receiving an audio input associated 
with the subject so as to generate audio data (see element 106, figure 3; paragraph 
0036) (note that the digital cameras comprising a microphone for capturing an image, 
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and receiving an audio input for generating image and audio data are well known in the 
art and are admitted as prior art by the applicant in paragraphs 0005 and 0045); and 
printing a picture of the subject responsively to the image data (see element 128, figure 
3; paragraph 0036-0037), while encoding the audio data using markings (i.e. dots, see 
paragraph 0038) in the printed picture that are substantially imperceptible to an unaided 
eye of a human viewer (see paragraph 0037). 

Re claim 4, Bhaskaran further discloses capturing the image comprises 
photographing the image using an electronic imaging camera (see paragraph 0036), 
and wherein receiving the audio input comprises recording the audio input using a 
microphone coupled to the camera (see paragraph 0036) (Note that the digital cameras 
comprising a microphone for capturing an image, and receiving an audio input for 
generating image and audio data are well known in the art and are admitted as prior art 
by the applicant in paragraphs 0005 and 0045). 

Re claim 5, Bhaskaran further discloses printing the picture comprises printing a 
halftone picture comprising dots of varying sizes within respective cells (see paragraphs 
0034-0035), and wherein encoding the audio data comprises varying respective 
positions of the dots within the cells responsively to the audio data (see figure 9, and 
paragraphs 0048-0052). 

Re claim 6, Bhaskaran further discloses detecting and decoding (i.e. extracting) 
the markings (i.e. audio data) in the printed picture (see paragraph s 0013-0017, and 
0036), and generating an audio output responsively to the decoded markings (see claim 
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2, and paragraph 0036, note that as mentioned in paragraph 0036 that the "Display 
device 116b represents a display device configured to display a hardcopy, e.g., a printout, of 
audio augmented image data 122, wherein the audio data is visually imperceptible" and "Audio 
augmented image data 122 is then transmitted to a display device for presentation or printout. 
Display device 116a represents a display device configured to display a softcopy, e.g., an 
electronic copy viewable on a display screen while the audio data is played back, of audio 
augmented image data 122", paragraph 0036). 

Re claim 8, Bhaskaran further discloses scanning a picture comprising an image 
and incorporating in the image markings that are substantially imperceptible to an 
unaided eye of a human viewer (see element 238 in figure 12, and paragraphs 0034- 
0038) and that encode audio data associated with the image; detecting and decoding 
the markings in the scanned picture (see element 238 in figure 12, and paragraphs 
0034-0038 & 0066-0067); and generating an audio output responsively to the decoded 
markings (see claim 2, and paragraph 0036, note that as mentioned in paragraph 0036 
that the "Display device 116b represents a display device configured to display a hardcopy, 
e.g., a printout, of audio augmented image data 122, wherein the audio data is visually 
imperceptible" and "Audio augmented image data 122 is then transmitted to a display device for 
presentation or printout Display device 116a represents a display device configured to display a 
softcopy, e.g., an electronic copy viewable on a display screen while the audio data is played 
back, of audio augmented image data 122", paragraph 0036). 
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Re claim 9, Bhaskaran further discloses apparatus for encoding information (see 
figure 3), comprising: an image capture device (i.e. digital camera, see paragraph 
0036), which is arranged to capture an image of a subject so as to generate image data 
(see paragraph 0036); a processor, which is coupled to receive audio data associated 
with the subject (see paragraph 0036, note that it is inherent and well known in the art 
that the digital camera has a processor inside it for performing the audio and image 
related functions) (see also paragraph 0068), and which is arranged to generate a 
composite image (i.e. audio augmented image data) of the subject comprising the 
image data, while encoding the audio data in the composite image using markings that 
are substantially imperceptible to an unaided eye of a human viewer (see paragraphs 
0036-0037); and a printer (see element 128, figure 3), which is arranged to print a 
picture of the subject comprising the encoded audio data responsively to the composite 
image (see figure 3, and paragraphs 0036-0037). 

Re claim 10, Bhaskaran further discloses the image capture device (i.e. digital 
camera) comprises an electronic imaging camera (i.e. digital camera), which further 
comprises a microphone for capturing the audio data (see paragraphs 0036 & 0066, 
note that it's inherent that the camera has a microphone in order for the camera to 
capture the audio data). 

Re claim 1 1 , Bhaskaran further discloses the picture comprises a halftone picture 
comprising dots of varying sizes within respective cells (see paragraphs 0034-0035, 
0039), and wherein the processor (see figure 3) is arranged to vary respective positions 
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of the dots within the cells so as to encode the audio data (see figure 9, and paragraphs 
0039, 0048-0052). 

Re claim 12, Bhaskaran further discloses a scanner (see paragraph 0038), 
which is arranged to detect the markings in the printed picture (see figure 4, and 
paragraph 0038), so as to permit an audio output to be generated responsively to the 
markings (see claim 2, and paragraph 0036, note that as mentioned in paragraph 0036 
that the "Display device 116b represents a display device configured to display a hardcopy, 
e.g., a printout, of audio augmented image data 122, wherein the audio data is visually 
imperceptible" and "Audio augmented image data 122 is then transmitted to a display device for 
presentation or printout Display device 116a represents a display device configured to display a 
softcopy, e.g., an electronic copy viewable on a display screen while the audio data is played 
back, of audio augmented image data 122", paragraph 0036). 

Re claim 14, Bhaskaran further discloses a scanner (see paragraph 0038), which 
is arranged to scan a picture comprising an image incorporating markings that are 
substantially imperceptible to an unaided eye of a human viewer and that encode audio 
data associated with the image (see element 238 in figure 12, and paragraphs 0034- 
0038, 0066-0067); a processor (see figure 3), which is arranged to detect and decode 
the markings in the scanned picture so as to recover the audio data from the picture 
(see element 238 in figure 12, and paragraphs 0034-0038, 0066-0067); and an audio 
speaker, which is coupled to the processor so as to play the recovered audio data (see 
paragraphs 0036 & 0066, note that it's inherent that the microphone is coupled to the 
processor in order for the playback the audio data, also note that as mentioned in 
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paragraph 0036 that the "Display device 116b represents a display device configured to 
display a hardcopy, e.g., a printout, of audio augmented image data 122, wherein the audio data 
is visually imperceptible" and "Audio augmented image data 122 is then transmitted to a display 
device for presentation or printout. Display device 116a represents a display device configured 
to display a softcopy, e.g., an electronic copy viewable on a display screen while the audio data 
is played back, of audio augmented image data 122", paragraph 0036) (see also paragraph 
0068). 

Regarding claims 15-18, they are interpreted and thus rejected for the reasons 
set forth above in the rejection of claims 3, 5-6, and 8 respectively, since claims 15-18 
disclose a computer readable medium of instructions for carrying out the method (a 
software product) that corresponds to the encoding and recovering information method 
of claims 3, 5-6, and 8, thus the computer implemented method is inherent and 
arguments made for claims 3, 5-6, and 8 are applicable for claims 1 5-1 8 respectively. 

Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

6. Claims 7 & 13 are rejected under 35 U.S.C. 103 as being unpatentable over 
Bhaskaran et al., US 2004/0141630 in view of Saindon et al., US 2002/0161579 
further in view of Lenir et al., US 2004/0107107. 
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Re claim 7, Bhaskaran further discloses the audio input comprises speech (see 
paragraph 0049, note that human voice recorded by the camera microphone is referred 
as speech). 

Bhaskaran fails to disclose that receiving the audio input comprises converting 
the speech to at least one of text and prosody of the speech, and wherein encoding the 
audio data comprises encoding the at least one of the text and the prosody. 

Saindon et al. discloses that receiving the audio input (see abstract) comprises 
converting the speech to at least one of text (see abstract and paragraph 0012), and 
wherein encoding the audio data comprises encoding the at least one of the text (see 
paragraph 0024-0028). 

Lenir et al. discloses converting the speech to prosody (i.e. utterance) of the 
speech (see abstract and paragraph 0051), and wherein encoding the audio data 
comprises encoding the prosody (see abstract) 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention to modify the method and apparatus for augmenting a digital image 
with audio data as disclosed by Bhaskaran to include the speech to text converter as 
taught by Saindon and speech processing process as taught by Lenir in order to 
"provides audio to text conversion with high accuracy in short periods of time" as taught 
by Saindon in paragraph 0008, and to provide a method for processing speech 
utterance as taught by Lenir in abstract & paragraph 0008. 
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Re claim 13, claim 13 recites identical features, as claim 7, except claim 13 is an 
apparatus claim. Thus, arguments made for claim 7 are applicable for claim 13. 

Conclusion 

The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

Mclntyre et al, US 6,102,505, see whole document. 
Kanevsky et al., US 6,687,383, see whole document. 
Bhaskaran et al., US 6,064,764, see whole document. 

Contact Information 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Pawandeep S. Dhingra whose telephone number is 
571-270-1231 . The examiner can normally be reached on M-F, 9:30-7:00. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Twyler Lamb can be reached on 571-272-7406. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
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Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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